Web Metasearch as Belief Aggregation

Web metasearch requires a mechanism for combining rank-ordered lists of ratings returned by multiple search engines in response to a given user query. We view this as being analogous to the need for combining degrees of belief in probabilistic and uncertain reasoning in artificial intelligence. This paper describes a practical method for performing web metasearch based on a novel transformationbased theory of belief aggregation. The consensus ratings produced by this method take into account the item ratings/rankings output by individual search engines as well as the user’s preferences. Copyright (~) 2000, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. Introduction Web search engines (WSE) use tools ranging from simple text-based search to more sophisticated methods that attempt to understand the intended meanings of both queries and data items. There has been much work in this area in recent years. The link structure of the web has been used to understand the relationships between documents (Chakrabarti et al. 1999). Machine learning techniques have been applied to web search (McCallum et al. 1999), (Boyan, Freitag, & Joachims 1996). Specialized agents that mine the web have been described (Doorenbos, Etzioni, & Weld 1997). Light is shed on web search from a different perspective by work on human behavior (Macskassy et al. 1998). Related problems include those of intelligently recommending scientific papers (Basu et al. 1999) and creating digital libraries for efficient indexing and retrieval Of scientific documents (Lawrence, Bollacker, & Giles 1999). Reviews of work in web searching include (Lawrence & Giles 1999), (Filman & (guest editors) 1998), (Lawrence & Giles 1998). We are interested in web metasearch engines (MSE) (Selberg & Etzioni 1995), (Glover et al. 1999), which dispatch user queries to several available WSE; each WSE produces an ordered list of data items in response to the query, and the MSE combines these lists into a single summary list that is then passed on to the user. In the present paper we present a new approach to web metasearching. Numerical relevance ratings are provided as part of our method’s output. A useful feature of our approach is that it allows the user to give subjective confidence values for the particular WSE being employed. These confidence values determine the relative importance accorded to the different WSE when producing the final search summary that the user receives as output. Our approach is based on a framework (Alvarez 1997), (Alvarez 2000) provides a set of tools with which to systematically construct combination operators for belief aggregation, each determined by a different choice of geoFrom: AAAI Technical Report WS-00-01. Compilation copyright © 2000, AAAI (www.aaai.org). All rights reserved. metric transformation or reference frame in an abstract space (in the present case this space is the space of relevance ratings). Combination operators allow one to uniformly assign numerical relevance ratings to the items found by the WSE being polled by the system, and thus ultimately to produce the final summary list. Our approach assumes that the WSE return numerical ratings in addition to rankordered lists of hits. If this is not the case, ratings may be assigned to rankings in some way before combination is to be performed. Algorithms for combining rankings when numerical ratings are not available have been studied previously, e.g. (Freund et al. 1998). High flexibility and configurability are two properties that our approach inherits from the theoretical framework of (Alvarez 2000). Our framework provides a natural mechanism to vary the sensitivities of the resulting combination operators to their various inputs. This allows a system based on this approach to adapt according to the user’s preferences.